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ABSTRACT 

The purpose of this thesis is to determine if format 
variation has any affect on speed and accuracy of data 
entry for the Marine Integrated Fire and Air Support System 
(MIFASS) . 

Independent variables for the test are message formats, 
message types, and subjects. Subject learning effects and 
experimenter differences are controlled by use of a pilot 
study. Subjects are included as a treatment in order to 
estimate their effects. 

The experiment is designed to support both parametric 
and nonparametric analyses. A 4x5x8 randomized complete 
block factorial with two replications is used to analyse 
the effects of format variation. Friedman's Two-Way ANOV 
by Ranks is used to test subjective rankings of the formats 
as concerns ease of use and operational suitability. 

It is concluded that format variations do affect speed 
of data entry but not accuracy. Format 3 appears to be 
superior. Recommendations for further study are given. 
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I. 



STATEMENT OF THE PROBLEM 



A. THE PROBLEM IN GENERAL 

The primary function of a Fire Support Coordination 
Center (FSCC) at any level of command is to provide the 
commander with the means for planning and coordinating 
available fire support. Specific responsibilities of a 
FSCC include fire support planning, designation of targets 
and assignment of priorities, allocation of fire support 
resources, and coordination of supporting arms with each 
other and the supported force. At present, all these 
responsibilities and functions are carried out manually 
through the use of radio and telephone communications, 
personal liaison, and an immense amount of paperwork. An 
inordinate amount of valuable time is lost in the FSCC 
during normal operations because of the paperwork and 
liaison required to accomplish its mission. In an attempt 
to eliminate this bottleneck and more effectively use the 
skill and training of the personnel who operate the FSCC, 
the Marine Corps, in conjunction with Hughes Aircraft 
Company, is developing an automated system which will 
materially reduce the paperwork, list keeping, and manual 
labor presently required in a FSCC. The system will be 
called the Marine Integrated Fire and Air Support System 
(MIFASS) . 
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B. OVERVIEW OF THE AUTOMATED FSCC 



The concept for MIFASS operations offers extensive 
improvement in tactical effectiveness through automation of 
fire request planning, fire support coordination, fire and 
air support planning, conflict prediction, and the consoli- 
dation of many other common functions at the MIFASS centers. 
The automated system will utilize various configurations of 
the hardware listed in Table I at designated levels of the 
operational tactical unit. Subunits operating under tacti- 
cal control of the MIFASS centers will input intelligence 
data, fire support requests, med-evac requests, damage 
assessment reports, etc., to the MIFASS computer. MIFASS 
software will provide rapid response to many on-line users 
by interleaving the processing tasks and overlapping the 
input/output (I/O) demands with processing. Through 
generalized data management software, the system will permit 
definition of new messages, files, reports, and displays in 
the field without reprogramming. The application programs 
envisioned will include computer assistance and algorithm 
development for major requisite functions such as fire 
mission analysis, target data listing, air support control, 
technical fire control, conflict detection, troop safety, 
mission scheduling, resource allocation, etc. Since each 
FSCC will have access to the memory bank and coordinating 
agencies will be able to view the same data simultaneously 
on their own output units, the time needed to perform the 
required functions for all subunits will be materially 
decreased thereby increasing the effectiveness of the 
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overall system. Due to the probability that several agencies 
will be viewing the same data simultaneously and in the 
interests of simplification in training and programming, 
it is evident that a simplified and uniform data I/O pro- 
cedure will be required. 



TABLE I 

HARDWARE REQUIRED FOR A FIELDED MIFASS 

Each FSCC will employ a specific configuration of some or 
all of these units. 

Digital Message Entry Device (DMED) 

Portable Printer 

Remote Terminal Device (RTD) 

Cathode Ray Tube (CRT) Display 
Small Auxiliary Memory 
Hard Copy Printer 
Keyboard Entry Panel 
Communications Net 

Computer with Internal Memory 16 -13 IK 

Auxiliary Mass Memory 500K-8M Characters 

Magnetic Tape Units 

Printers 

Plotters 

Large Screen Displays 
Textual/Tabular Displays 
Graphic Displays 



11 






■mm- 



C. DATA ENTRY PROCEDURES 



Proper data I/O procedures are crucial to the function- 
ing of an automated system. One phase of these procedures 
which presents a potential bottleneck to the system is the 
format used when entering data or outputting it on a grid 
display or copier for viewing by interested parties. If a 
uniform format is not used by everyone, users will encounter 
problems in data retrieval and in interpretation of data 
entered by other on-line units. The requirement for a 
format (or formats) understood by all which is fast, accu- 
rate, and convenient is obvious. Determining the feasi- 
bility of such a format was the overall objective of this 
experiment. 

D. PURPOSE OF THE TEST 

As one approach to determining the feasibility of a uni- 
form format for use in the system, this test was designed 
to evaluate differences in operator data entry speed and 
accuracy resulting from the use of alternative message 
formats when used with differing types of I/O data. The 
convenience and operational suitability of the formats 
tested was to be determined through operator surveys. 

E. HYPOTHESES THAT WERE TESTED 

^ol* There is no significant difference in the speed of 
data entry due to the use of different message formats. 

H^2* There is no significant difference in the accuracy 
of data entry due to the use of different message formats. 
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There -is no significant difference in the speed of 
data entry due to the use of different message types. 

There is no significant difference in the accuracy 
of data entry due to the use of different message types. 

There is no significant difference in the speed of 
data entry due to the subjects used for the test. 

There is no significant difference in the accuracy 
of data entry due to the subjects used for the test. 

There is no significant interaction between message 
type and format for speed of data entry. 

H^gt There is no significant interaction between message 
type and format for accuracy of data entry. 

: There is no significant interaction between message 

format and subject for speed of data entry. 

H There is no significant interaction between 

olO 

message format and subject for accuracy of data entry. 

There is no significant interaction between 
message type and subject for speed of data entry. 

There is no significant interaction between 
message type and subject for accuracy of data entry. 

There are no significant interactions among 
message type/ message format, or subject for speed of data 
entry . 

There are no significant interactions among 
message type, message format, or subject for accuracy of 
data entry. 

The subjects are indifferent to the message for- 
mats concerning ease of use. 
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The subjects are indifferent to the message for- 
mats concerning operational suitability. 



F. ASSUMPTIONS 

1 . Message Types 

The message types selected for this experiment were 
assumed representative of the total population of messages 
envisioned for use in an operational MIFASS. The messages 
chosen varied in length from 62 to 573 characters which were 
a mixture of numeric and alpha-numeric; therefore, they 
covered a wide span of the total population. 

2 . Subjects 

The subjects used for this test were assumed repre- 
sentative of the total population of personnel available to 
operate a fielded system. This assumption might be ques- 
tioned in view of the experience and proficiency gained by 
the subjects during the preceding several months of system 
development. However, circumstances dictated that these 
particular subjects be used, and, in any case, it is a 
certainty that personnel assigned to operate a fielded 
system will undergo several weeks of intensive training 
before being permitted to begin actual operational duties. 
Thus, this assumption was considered justified. 

3 . Message Format Effects 

Effects of different message formats on the speed 
and accuracy of data entry under test conditions were con- 
sidered typical (ignoring stress, fatigue, etc.) of the 
effects which will be prevalent during actual operations. 
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If it is subsequently determined that any conditions make 
this assumption invalid, this experiment can be repeated 
under those conditions during later phases of MIFASS 
development . 

G. GENERAL APPROACH 

1 . The Test 

Four test runs were made, each of which consisted of 
a group of subjects entering data from five message types 
into a specific blank message format which was presented on 
a grid display screen. Each subject entered data twice for 
each message type for a total of 10 trials per run per 
subject. Speed and accuracy measures were taken for each 
trial. Upon completion of each run, the subjects each com- 
pleted a subjective evaluation form concerning the ease of 
use and operational suitability of that format. 

2 . The Analysis 

A randomized block factorial analysis of variance 
was used to test the first 14 hypotheses. The two subjec- 
tive hypotheses were tested using nonparametric test 
procedures . 

3 . Remarks 

It was not the objective of this experiment to choose 
a single format for use in a fielded system but to test the 
feasibility of such a format. However, if one of the for- 
mats tested were to be found faster and more accurate than 
the others under all test conditions, it might be assumed 
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that that format would be a good possibility for adoption 
as a universal format for the MIFASS. 
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II. SELECTION OF DESIGN 



A. DEPENDENT VARIABLES 

1 . Speed of Data Entry 

The time, T^ , required to enter each stimulus 
message into a blank message format was collected for each 
subject during each experimental run. This response was 
then normalized by dividing each T^ by the total number of 
keying actions required for that message format including 
any header data or spaces over which the operator had to 
space or tab manually. Thus, the dependent variable speed 
in this test was the normalized data entry time. 

2 . Accuracy of Data Entry 

The accuracy of data entry was defined in terms of 
data elements which are entire sets of one or more charac- 
ters to be entered into a single message field. The number, 
A^ , of data elements that had been entered incorrectly 
was collected for each message entered by each subject dur- 
ing each experimental run. Any error such as single or 
multiple' typographical errors within a data element, an 
omitted data element, or one data element substituted for 
another was counted. This response was normalized by 
dividing A^ by the total number of data elements that were 
entered in the i^^ message. Thus, the dependent variable 
accuracy in this test was the normalized data entry accuracy 
score . 
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3. Subjective Rankings 



To forestall the possibility that individual memories 
might affect the ranking of the formats if the subjective 
opinions were to be gathered after all the formats had been 
presented, it was considered necessary to gather a separate 
opinion from each subject for each format immediately after 
it had been tested. A graduated interval scale was designed 
for each subjective dependent variable. Each scale con- 
sisted of 20 intervals with the endpoints and some inter- 
mediate intervals marked with verbal descriptions for ease 
in marking as shown in Fig. 1. These scales were then used 
for gathering the individual opinions. 

a. Ease of Use 

The graduated scale used by each subject to give 
his opinion of the ease of use of each format was marked 
from 0 to 20 where 0 coincided with the rating "Easy to Use" 
and 20 matched "Unusable." 

b. Operational Suitability 

The opinions for this subjective dependent 
variable were marked on a graduated scale where 0 coincided 
with "Very Suitable" and 20 with "Very Unsuitable." 

B. INDEPENDENT VARIABLES 
1 . Message Formats 

The independent variable of primary interest was 
message format. There were four different formats tested 
in this experiment. Appendix A presents the actual formats 
tested for each message type. 
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SUBJECT REACTION FORM 



Siabject Format 

Experimenter Date 

Please indicate your reaction to the format you have just 
completed testing by placing a checkmark at the appropriate 
place on the line below. 

1. How easy or difficult was it to use this format? 

0 5 10 15 20 

• ■ ■ I I < ■ I ■ I I , > I 1 i,,,i I I I I I I I I 

Easy Convenient Useable May be Unuseable 

to Use Useable 



2. How would you rate this format as far as suitability 
for use in an operational MIFASS? 

0 5 10 15 20 

« » » « * * « » * * * « j- - I i 1 1 \ I t 

Very Suitable Possibly Unsuitable Very 

Suitable Suitable Unsuitable 



Figure 1. Subject Reaction Form. This form was used to 
gather data necessary to the nonparametric 
analyses required. 
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a. Linear Format 



In this format, each field name in the message 
was immediately followed, on the same line, by a space or 
spaces in which the pertinent message data for that field 
was to be entered. One field followed the previous field 
on the same line without spacing between fields until that 
line was filled. The succeeding fields were positioned in 
lines as stated until the entire message had been completed. 

b. Block Format 

The block format had the field names in a linear 
sequence with the message data space or spaces immediately 
below the field name on the next line of the display field. 
The field names were separated sufficiently to preclude 
information overlap in the message data spaces below. No 
space appeared between the line of field names and the line 
containing related message data. A double space appeared 
between each adjacent pair of lines. This form was followed 
until enough line pairs appeared to complete the entire 
message. 

c. Columnar/Separated Format 

For this format, all field names appeared in a 
single column on the left side of the display area. The 
space or spaces for the pertinent message data for each 
field appeared in a second column indented from the left 
margin of the display area, not from the end of the field 
name. Thus, the message data column appeared as a distinct 
column with a uniform margin. 
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d. Columnar/Run-On Format 

This format had the field names in a column as 
above, but the space or spaces for message data appeared 
without spacing to the immediate right of the field name. 
Instead of a single column of field names-message data as 
used above, three columns were used where space on the 
display area existed; otherwise, two columns v;ere used. 

2 . Message Types 

An independent variable of secondary interest was 
message type. As previously discussed, the five types 
selected for this experiment were assumed to be representa- 
tive of the total population of messages to be found in a 
fielded system. Data pertinent to these message types is 
shown in Table II. The actual messages used form Appen- 
dix B . 



TABLE II 

DATA ON MESSAGE TYPES 



Msg. 

Type 


Msg. 

Name 


Msg. Name 


Number of 
Characters 


Number of 
Fields 


1 


CHKFIM 


Check Fire Command 


62 


6 


2 


EOMBDM 


End of Mission Battle 
Damage Assessment 


96 


10 


3 


TGTLST 


Target List Input 


151 


14 


4 


CFFTGR 


Call for Fire by 
Target Number 


266 


23 


5 


FORDRJl 


Fire Order Universal 


573 


54 
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3 . Subjects 



The other independent variable of secondary interest 
was the subjects participating in the test. As discussed, 
these subjects were assumed to be representative of the 
total population of potential system operators. However, 
their previous training and experience might have affected 
their performance and might have introduced bias into the 
experiment. The inclusion of subjects as treatments per- 
mitted an analysis of their effects on the dependent vari- 
ables to be made. Eight subjects were used. 

4 . Replications 

The independent variable of tertiary interest was 
replication. The inclusion of this variable was principally 
to provide a check to see if the experiment was conducted 
consistently . 

C. CONTROLLED VARIABLES 

There were several nuisance variables which had the 
potential to bias the outcome of the experiment. Subjects 
were such a potential source of bias and their effects were 
controlled by including them as a treatment in the experi- 
mental design. Other nuisance variables could not be exper- 
imentally controlled as were subjects but had to be con- 
trolled by other means. 

1 . Abbreviations and Acronyms 

This source of bias was controlled both experimentally 
and statistically. The use of abbreviations and acronyms in 
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all message types is not necessarily proportional to the 
length of the message; therefore, the normalization of the 
dependent variables as discussed in Para. II. A was also 
utilized to statistically control this effect by keeping 
the variation more nearly constant over the normalized data 
base. Standard abbreviations and acronyms were used for 
all message types for all formats. This assumption held 
these effects constant. 

2 . Experimenter Differences 

Two experimental controls were used to minimize 
effects due to potential experimenter differences. First, 
all experimenters were trained to administer the experiment 
in a standard fashion thus minimizing effects due to differ- 
ent testing procedures. Second, each experimenter presented 
each format twice and administered no more than one test 
run to any given subject. 

3 . Subject Learning Effects 

To minimize learning effects, two experimental con- 
trols were used: training and randomization of presentation 

of formats. All subjects completed the Marine Tactical 
Command and Control System (MTACCS) Test Bed Operations 
Training Course prior to being administered the test. Such 
training ensured familiarity with all five message types 
and all four formats. Thus, when proceeding from a test 
run with one format to the next run with another, a subject's 
learning effects were minimized. Remaining effects which 
could possibly introduce bias were reduced by randomizing 
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the order of presentation of the formats and message types 
to the subjects. 

4 . Stimulus Presentation 

a. Message Input Stimuli 

The stimulus messages were presented in hard 
copy form only. Radio or voice presentation was deemed too 
difficult to control with the facilities available for the 
test. See Appendix B. 

b. Format Stimuli 

The formats used for this test were presented 
to the siabject on the grid display screen. One format was 
presented at a time in the order shown in Table III. The 
random ordering of the presentation of each format minimized 
any possible effects from subject learning that could have 
occurred if the formats had been presented to all the sub- 
jects in the same order. 

D. STATISTICAL DESIGN 

Since it was desired to secure data from every combin- 
ation of the independent variables, a randomized complete 
block factorial design, 4x5x8 with two replications, was 
selected for this experiment. This design required 320 
data points with which to test the hypotheses and provided 
an extremely high power for the test. Table III presents 
the test schedule required to support this design. 
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TEST SCHEDULE 
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TABLE III (Cont.) 



Sj Sg Sg 

FTFTFTFTFTFTFTFT 



Run 

4 



4 





2 

5 

3 
1 
5 

4 
4 
1 
3 
2 



1, Assumptions 
a. Model 

The mathematical model assumed for this design 
was a fixed factor model as developed in Ostle [Ref. 1]: 

X . . , = U+R . +A . +B, +C-, + ( AB ) • 1 + (AC) . ■■ + (BC ) , 

ijkl 1 j k 1 jk jl kl 



+ (ABC) 



where X. . 

ijkl 

u 




observation ijkl 

mean effect (constant) 

effect of replicate i, i=l,2 

effect of message format j, j=l,2,3,4 

effect of message type k, k=l,2,3,4,5 
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= effect of subject 1, 1=1, 2,..., 8 

(AB) etc. are interaction terms 

= random error for observation ijkl 

It was assumed that was distributed normally with mean 

2 

zero and variance a where e is homoschedas tic for all i,j, 
k, and 1. It was further assumed that there were no repli- 
cation by treatment interactions and that the observations 
were independently distributed multivariate normal. 

b. Treatment Effects 

As discussed, all factors in this model were 
assumed to be fixed. There is no doubt of this for the 
formats and message types; however, the subject sample 
might be argued to be random rather than fixed. Consider- 
ing the experience gained by all the subjects through 
several months familiarization with the test bed and its 
equipment and their overall level of past Marine Corps 
operational experience and training, it was decided that 
generalization of the conclusions made from this test over 
the entire population of possible field operators would not 
be totally valid. Therefore, of necessity, this factor had 
to be considered fixed, also. 

c. Effects of Nonconformity to Assumptions 

In general, the consequences are not serious 
when the assumptions made in connection with the analysis 
of variance are not strictly satisfied. For example, minor 
deviations from normality and/or some degree of heterosche- 
dasticity will have little effect on the usual tests and 
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the resulting inferences. Reference 2 discusses this aspect 
of experimental design thoroughly. 

2 . Sample Size 

To determine the proper sample size to use for this 
test, one has to use ideas developed by Tang [3] and used 
by Pearson and Hartley [4] in the development of their 
power function curves. Using these developments, one cal- 
culates the sample size required to yield a specified power 
for the desired test utilizing this equation: 




o//n~ 

where (f) = power function 

th 

D. = difference between the effect of the i 

treatment and the mean effect of all treatments 

k = number of treatments (i.e., number of levels 
of the treatment in question 

a = standard deviation of errors 

n = number of observations required for each 
treatment level 

Since, for this experiment, a and were not known, a modi- 
fication to this equation was made to circumvent these 
problems. It was assumed that a maximum detectable treat- 
ment difference of one a about the mean would be sufficient 

2 

for this experiment. It can be shown that the minimum 2^0^^ 
occurs when two of the treatment differences are - ^ and 
j and the remaining differences are 0 [Ref. 7] . Then 
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+ 0 + 



+ 0 



o 



2 



2 



Hence, if the maximum detectable difference is other than 
that chosen, the power of the test will increase. Now, the 
above equation for (J) becomes 



This form was much simpler to utilize and was no longer 
sensitive to errors in estimating a. Using this power 
function with the proper degrees of freedom and a = 0.05, 
the available power can be determined for various sample 
sizes from the curves produced by Pearson and Hartley [4] . 
As shown in Table IV, a power of approximately 0.90 was 
attainable for the test of format differences with as few 
as three subjects, each subject using two samples of each 
message type for each format; whereas, four subjects were 
required to get a similar power for the test of message 
types. Since there were 40 observations for each subject, 
effects of subject differences could be tested with 
extremely high power for almost any number of subjects. 
However, as the use of eight subjects did not add mater- 
ially to the cost or time required but did enhance the 
power of any test desired, eight subjects were used. 
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TABLE IV 





AVAILABLE 


POWERS FOR THE 


TEST 




No. of 
Subj . 


Obs./Subj ./ 
Cell 


Degrees of 
Freedom for 
Error Term 


Power 




Format 


Msg . 
Type 


12 


2 


239 


0.999 


0.999 


10 


2 


199 


0.999 


0.999 


8 


2 


159 


0.999 


0.999 


6 


2 


119 


0.999 


0.982 


4 


2 


79 


0.972 


0.890 


4 


3 


158 


0.999 


0.984 


3 


2 


59 


0.895 


0.770 


3 


3 


118 


0.986 


0.950 


2 


2 


39 


0.710 


0.550 


2 


3 


78 


0.910 


0.790 


2 


4 


117 


0.968 


0.900 
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III. LOGISTICS AND TRAINING 



A. SUBJECTS 

The eight subjects used for this test were Marine Corps 
officer and enlisted personnel assigned to the MTACCS Test 
Bed which has direct responsibility for developing MIFASS. 
Names of 22 eligible officers and men formed a pool from 
which eight were drawn randomly to act as subjects. Pre- 
requisites for eligibility were previous training or 
experience in fire support coordination, artillery coordi- 
nation, target data analysis, and test bed operations. A 
capability of an extremely high or low typing speed rendered 
potential subjects ineligible as it was desired to have 
typing speed on a level representative of personnel who will 
serve in a fielded MIFASS. Each subject had completed the 
MTACCS Test Bed Operations Course which covered those 
subjects listed in Table V. Prior to their first test run, 
the subjects were thoroughly briefed on the experiment. 

B. EXPERIMENTERS 

The four experimenters necessary for this test were 
selected similarly to the subjects except that all test bed 
personnel, including the civilian contractor employees, 
were considered eligible for duty as experimenters. Avail- 
ability was the prime consideration in experimenter selec- 
tion. Those selected were familiar with MTACCS Test Bed 
operations through the above mentioned course and were all 
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TABLE V 



SUBJECTS COVERED IN THE MTACCS 
TEST BED OPERATIONS COURSE 



MTACCS Concept 

Role of the Test Bed 

MIFASS TSM Concept 

Concept of Test Bed Operations 

Display and Entry Devices (I/O) 
(Included practical exercises) 



Marine Corps personnel. Additional training included 
experiment familiarization sessions conducted by the test 
administrator, practice sessions, and a pilot study as 
discussed in Para. IV. A. Those selected also attended the 
subject briefings. 

C. HARDWARE AND SOFTWARE REQUIREMENTS 

1 . Hardware 

The hardware listed in Table VI was utilized for 
the performance of this experiment. 

2 . Software 

The software used for this experiment is discussed 
thoroughly in Ref. 5. Since it is not in the public domain 
nor pertinent to a general discussion of the test, a listing 
is omitted here. 
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TABLE VI 



HARDWARE REQUIRED FOR THIS EXPERIMENT 



Unit 


Model 


Quantity 


CDC Computer System 


3300 


1 


CDC Grid Display System 


243-1 


1 


Call Director 


600 


2 


Voice Recorder 


TR-1720 


1 


Time Code Generator 


TC-201 


1 



D. TEST EQUIPMENT AND MATERIAL 

1 . Subject 

In order for the subjects to function efficiently, 
they had at their disposal on the grid display console a 
reference chart and a MTACCS Glossary. The reference chart 
contained field header data and allowable field data entries 
used in all MIFASS messages and displays. The MTACCS 
Glossary contained all the abbreviations and acronyms to be 
used. 

2 . Experimenter 

Each experimenter had at his disposal the following 
items: a stopwatch. Parametric Data Collection Forms, 

Subject Reaction Forms, the voice recorder, and the input 
message stimuli. 
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E. STIMULUS MATERIAL 

All stimulus materials were developed by Hughes Aircraft 
Company in conjunction with Test Bed officers. Message 
input stimuli for the five messages used appear in App. B. 
The formats presented to the subjects for each message 
appear in App. A. 

F. DATA COLLECTION FORMS 

To support the analyses necessary for this test, two 
data collection forms were necessary. 

1 . Parametric Data Collection Form 

As depicted in Fig. 2, this form was used to collect 
data for use in determining speed and accuracy measures . 

One form was completed by the applicable experimenter for 
each subject for each format. A total of 32 forms were 
required . 

2 . Subject Reaction Form 

As discussed earlier, this form was necessary as an 
intermediate step in the nonparametric analysis. A total of 
32 forms were required as each subject completed one for 
each format immediately upon completion of the run testing 
that format. Figure 1 depicts this form. 
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Subject 




Date 




Format 




Experimenter 


Message Type 


Time 


No. of Errors 


Remarks 










Experimenter Notes 



Figure 2. Parametric Data Collection Form. 

This form was used to gather 
speed and accuracy data for the 
ANOV. 
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IV. TEST PROCEDURE 



A. PILOT STUDY 

1. Purpose 

The pilot study was primarily conducted in order to 
validate the test design, determine if the desired data 
could be collected, and see if the required analyses could 
be made using this data. Secondary considerations were 
training of experimenters, refinement of data collection 
procedures and forms , and detection and correction of errors 
in the test stimuli. 

2 . Administration 

a. Materials 

The test stimuli, equipment, and data collection 
forms used were those to be used during the actual test. 

b. Procedure 

The test administrator selected those indivi- 
duals to be used as experimenters during the test. Trial 
runs were made with each selectee acting as the subject to 
familiarize him with the entire test. Each selectee then 
performed as an experimenter for other runs using another 
experimenter as a subject. Data was collected as it was to 
be collected during the actual test. This procedure was 
followed until the test administrator felt that the level 
of experience was great enough to minimize any effects due 
to experimenter differences. 
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c. Results 



A trial analysis was made using the data 
gathered during the pilot study. The test design was 
confirmed as was the suitability of the data collection 
forms . 

B. TEST PROCEDURE 

The actual test runs were conducted as follows: 

Prior to each run, the test administrator ensured 
that the system was operating properly and that the correct 
software was loaded. 

The experimenter presented the subject with the 
proper message stimuli and recorded the event on the voice 
recorder. 

The format to be filled in by the subject was called 
to the screen from the memory. 

When the format to be tested appeared, the experi- 
menter recorded the event and started his stopwatch. 

The subject entered the necessary data using the 
alphanumeric keyboard on the grid console. 

Upon completion of his task, the subject pressed 
the STORE RECORD key. The experimenter stopped his stop- 
watch and recorded the time on the voice recorder and the 
Parametric Data Collection Form. 

The above steps were repeated until that run was 
complete. The subject then completed his Subject Reaction 
Form. 
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C. SUBJECT BRIEFING 



Prior to commencement of the first test run, a thorough 
briefing was given to all subjects and experimenters. The 
briefing served to acquaint the subjects with the test, its 
purpose, and its procedures. The test administrator pre- 
viewed the test, answered any questions the subjects or 
experimenters had, and provided each subject. with the test 
schedule. The subjects were cautioned not to discuss their 
reactions with other subjects until after completion of the 
entire test. 
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V. ANALYSIS 



A. PARAMETRIC ANALYSIS OF VARIANCE (ANOV) 

1 . Procedure 

a. Data Collection 

Test data for the dependent variables speed and 
accuracy were collected on the Parametric Data Collection 
Form. The time required to enter each message stimulus into 
each format by each subject was recorded by the experimenter 
but the accuracy observations were collected differently. 
Errors were detected through the use of a compare program 
by which the computer compared actual entries with the cor- 
rect entries and provided a printout of all entries con- 
taining errors. The Test Administrator then manually 
counted and recorded these errors. The data collected for 
both parametric analyses is presented in App. C. 

b. Normalization of Data 

In order to preclude effects due to the differ- 
ing lengths of the message types, the test data had to be 
normalized. 

(1) Speed . The normalizing factors for speed 
of data entry were the reciprocals of the number of Keying 
actions necessary to enter a message type into a particular 
format. For example, for Format 1 Message Type 1, the 
number of keying actions required was 26, so the time for 
each subject to enter the data for this combination was 
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divided by 26 to obtain the normalized times. These times 
appear in App. D. 

(2) Accuracy . The accuracy scores were norm- 
alized in a similar manner; however, the normalizing factors 
were the reciprocals of the number of complete data elements 
to be entered in each format for each message type. These 
normalized observations appear in App. D. 

c. Organization of Data 

In order to minimize the effort necessary for 
performing the ANOV for both speed and accuracy, the data 
in App. D was rearranged into interaction tables as shown 
in App. E [1] . 

d. Computations and ANOV Tables 

The computations necessary for obtaining the 
results to be entered in the ANOV tables were accomplished 
using the set of equations summarized in Ref. 1. The 
general form of an ANOV table is discussed in Ref. 1, also; 
therefore, it will not be presented here. 

2 . Results 

a. ANOV for Speed of Data Entry 

The ANOV table obtained for the dependent vari- 
able speed in this test is given in Table VII. The effects 
of the different format levels and message types are sig- 
nificant at a = 0.05 as are the interactions of format \\?ith 
both message type and subject. The acceptance or rejection 
of the hypotheses concerning speed of data entry is given 
below: 
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1 — 1 
0 


Rejected 


H , 
o3 


Accepted 


H c 
o5 


Rejected 


«o7 


Rejected 


»09 


Rejected 


»011 


Accepted 


«ol3 


Accepted 



b. ANOV for Accuracy of Data Entry 

Results obtained from the ANOV of the dependent 
variable accuracy are displayed in Table VIII. Significant 
effects are exhibited by formats, message types, and their 
interaction at a = 0.05. 

During the test, faulty design in the program 
for Message Type 2 entered in Format 1 required a data 
element to be carried over from one line to the next. The 
margin for the 80^^ character on the grid screen was not 
visible so several subjects had difficulty determining 
when to cease entering data on the first line and to con- 
tinue on the second. The higher incidence of errors for 
Format 1 and Message Type 2 made an adjusted ANOV for 
accuracy desirable. Therefore, another ANOV was conducted 
with the errors which were directly attributable to this 
programming problem omitted from the observations. Table IX 
displays the adjusted ANOV results. As shown, format and 
message type effects are no longer significant though 
their interaction remains so. For the two ANOV for accuracy 
of data entry, the acceptance or rejection of the hypotheses 
is : 
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TABLE VII 



ANOV 


TABLE : 


FOR SPEED OF 


DATA ENTRY 




Source 


df 


SS 


MS 


F 


Replications 


1 


. 096 


.096 


1.215 


Treatments 










Format (A) 


3 


2.330 


. 777 


9.835* 


Msg. Type (B) 


4 


.663 


. 166 


2.101 


Subject (C) 


7 


8.706 


1.244 


15.746* 


AB 


12 


2.552 


. 213 


2.696* 


AC 


21 


3.311 


. 158 


2.000* 


BC 


28 


1.871 


. 067 


. 848 


ABC 


84 


5.798 


.069 


. 873 


Error 


159 


12.613 


.079 




Total 


319 


37.938 






* 

Significant at a 


= 0.05 


. All others 


were tested 


against 


a = 0.05 and were 


not significant. 
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TABLE VIII 



ANOV 


TABLE FOR 


ACCURACY OF 


DATA ENTRY 




Source 


df 


SS 


MS 


F 


Replications 


1 


.00006 


.00006 


.03 


Tr atments 










Formats (A) 


3 


.02679 


.00893 


5.02* 


Msg. Types (B) 


4 


.03520 


.00880 


4.94* 


Subjects (C) 


7 


.01167 


.00170 


.96 


AB 


12 


.13266 


.01106 


6.21* 


AC 


21 


.03989 


.00190 


1. 07 


BC 


28 


.05343 


.00191 


1.07 


ABC 


84 


.11638 


.00139 


. 78 


Error 


159 


.28319 


.00178 




Total 


319 


.69927 







* 

Significant at a = 0.05. All others were tested against 
a = 0.05 and were not significant. 
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TABLE IX 



ADJUSTED ANOV TABLE FOR ACCURACY 
OF DATA ENTRY 



Source 


df 


SS 


MS 


F 


Replications 


1 


.00026 


.00026 


.17 


Treatments 










Formats (A) 


3 


.01105 


.00368 


2.44 


Msg. Types (B) 


4 


.00824 


.00206 


1.36 


Subjects (C) 


7 


.01844 


.00264 


1.75 


AB 


12 


.11210 


.00934 


6.19* 


AC 


21 


.03085 


.00147 


.97 


BC 


28 


.05538 


.00198 


1.31 


ABC 


84 


. 11288 


.00134 


. 89 


Error 


159 


.24037 


.00151 




Total 


319 


.58957 






* 

Significant at a = 
a = 0.05 and were 


0.05. All others were 
not significant. 


tested 


against 
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Original 


Adjusted 


«o2 


Rejected 


Accepted 


«o4 


Rejected 


Accepted 


H 

o6 


Accepted 


Accepted 


H o 
o8 


Rejected 


Rejected 


»ol0 


Accepted 


Accepted 


»ol2 


Accepted 


Accepted 


«ol4 


Accepted 


Accepted 



B. NONPARAMETRIC ANALYSIS 
1 . Procedure 

a. Data Collection 

Immediately upon completion of each test run 
each subject completed a Subject Reaction Form on which he 
marked his evaluation of the ease of use and suitability 
of the format tested. 

b. Analysis 

The Friedman Test is discussed thoroughly by 
Conover and Kirk [Refs. 6 and 7]. It requires two assump- 
tions (which are considered valid for this test) : (1) each 

subject's evaluation of the fou^ formats will be independent 
of every other subject's evaluation, and (2) a subject's 
evaluations may be ranked in some manner. The analysis for 
both ease of use and suitability is given in App. F. 
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2. Results 



a. Ease of Use 

The test statistic for this analysis was 15.96 
which, when compared with the 0.95 quantile of a Chi Square 
random variable with three degrees of freedom, is 
significant. 

b. Suitability 

For this analysis, the test statistic was 11.64 
which is also significant at the 0.05 level. 

c. Hypotheses 

Both H and H ,, were rejected at the 0.05 
ol5 ol6 

level, i.e., the subjects were not indifferent to the 
formats as hypothesized. 
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VI. CONCLUSION 



A. PARAI4ETRIC ANOV 

1 . Speed of Data Entry 

The results of the ANOV for this dependent variable 
clearly show that format variation does affect speed of data 
entry; thus, the primary purpose of this test was success- 
fully accomplished. However, due to significant inter- 
actions of the formats with both message types and subjects, 
interpretation of significant main effects must be qualified. 
Figure 3 illustrates the type of interaction detected between 
formats and subjects. 




Figure 3. Example of a Typical Format-Subject Interaction. 

It appeared that groups of subjects reacted in similar man- 
ners but that each group reacted differently from the other 
groups. For example, subjects 1,2,7, and 8 all reacted to 
the formats in the same order, i.e.. Formats 4,3,1, and 2 
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(from fastest to slowest), whereas. Subjects 4,5, and 6 
reacted in the order 3, 4, 2,1, and for Subject 3 the order 
was 2, 3, 1,4. No discernible pattern of subject character- 
istics such as I.Q., typing speed, test bed experience, MOS , 
etc., was evident within the groups. Though the subject 
levels were considered fixed, it may be concluded that no 
single format will be best for all the operators in a fielded 
system. For any format chosen, under the present concept, 
some operators will not be able to function optimally. This 
implies that rigid selection and training programs must be 
instituted in order to minimize the effects due to operator 
variations. The format-message type interaction implies 
that some message types are inherently faster (or slower) 
when entered in certain formats . 

2 . Accuracy of Data Entry 

The original ANOV clearly rejected the hypothesis 
that neither format nor message type would affect accuracy 
of data entry. However, when ignoring those errors directly 
attributable to the program error, an adjusted ANOV resulted 
in the acceptance of these hypotheses. The results of the 
adjusted ANOV indicate that these factors have little effect 
on accuracy of data entry; however, their interaction is 
significant. This implies that some of the message types 
are inherently more accurate (or inaccurate) when used with 
certain formats. 
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B. NONPARAMETRIC ANALYSIS 



Both hypotheses tested were rejected. The subjects 
tended to consistently rank the formats so that one or more 
were favored over the others. Viewing the totaled ranks for 
each format one may infer that the subjects tended to favor 
Format 3 over the others for both categories. In both cases 
the overall ranking of the formats was 3,4,1, and 2. 

C. SUMJ4ARY 

1. Purpose of the Test 

As discussed in Para. I, D, this test was designed to 
evaluate differences in operator data entry speed and 
accuracy resulting from the use of alternative message for- 
mats when used with differing types of I/O data. The test 
was not designed to determine which format would be best but 
to determine whether or not different formats would affect 
the dependent variables. To this extent, the purposes of 
this test were accomplished. Different formats do affect 
the speed of data entry but do not affect accuracy 
appreciably. 

2 . Inferences 

Due to the format-subject interaction, no one format 
can be inferred with certainty to be superior to the others. 
However, the mean speed for Format 3 was the smallest of 
the four formats tested and its error rate was very low. 

It received no rank below two for ease of use and was con- 
sidered first by four subjects and tied for first by one. 
Concerning operational suitability. Format 3 was ranked 
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first by four subjects and tied for first by one. It was 
definitely preferred over the other three formats tested by 
the majority of the subjects even though it was not fastest 
for several. Thus, Format 3 may be inferred, with reserva- 
tions, to be superior for use in a fielded system. 

D. RECOMMENDATIONS 

As shown, the evidence tended to place Format 3 over 
the other three formats for use in the MIFASS. Further 
testing should be contemplated to determine the reasons 
for subject and message type effects. If subject effects 
can be overcome through selection and training of operators , 
then further development can be directed toward determining 
optimum formats for the message types to be used. This 
study can be integrated into the ongoing development of the 
entire system if time does not permit other tests to be 
conducted alone and proper precautions are taken to prevent 
confounding of variables. Another aspect to be approached 
is the weighting of message types with regard to their 
importance to the success of the system. If a particular 
message type is crucial to the success of the system, it 
should have more weight when optimal formats are being 
considered. 
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FORMATS USED IN THIS TEST 
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ALT: ; TTYP : / ; SIZE 
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APPENDIX B: fffiSSAGE INPUT STIMULI 



MESSAGE TYPE 1 



Field Name 


Abbreviation 


Requested Entry 


Check Fire Msg. 


CHKFIM; 




From 


FR: 


AR61 


To 


TO; 


BC30 


Target Number 


TGT NO: 


BD2121 


Request Number 


RQNO: 


162310 


Date Time Group 


DTG: 


XXXXXXXX 


MESSAGE TYPE 2 






Field Name 


Abbreviation 


Requested Entry 


End of Mission 
Battle Damage 


EOMBDM; 




From 


FR; 


AA31 


To 


TO: 


BA20 


Target Number 


TGT NO; 


BBOlOO 


Request Number 


RQNO; 


210000 


Casualties 


CAS: 


00025 


% Destroyed 


%DEST: 


99 


Enemy Posture 


ENMY PSTR: 


Burn 


Date Time Group 


DTG: 


XXXXXXXX 
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MESSAGE TYPE 3 



Field Name Abbreviation Requested Entry 



Target List Input 


TGTLST; 




From 


FR: 


CA35 


To 


TO; 


AC40 


UTM Location 


GRID: 


DE64022610 


Altitude 


ALT: 


00045 


Target Type/Subtype 


TTYP: 


CEN /BN 


Size 


SIZE; 


100X100 


Attitude 


ATT: 


08 


Degree of Protection 


DOP; 


HV 


Quantity 


QTY: 


1 


Target Priority 


PRI: 


1 


Target Status 


STATUS : 


CONF 


Date Time Group 


DTG: 


XXXXXXXX 
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MESSAGE TYPE 4 



Field Name 



Abbreviation 



Requested Entry 



Call for Fire by 
Target Number 


CFFTGR; 




From 


FR; 


AG31 


To 


TO: 


AG20 


Target Number 


TGT NO: 


MT0732 


Target Type/Subtype 


TTYP : 


VEH /A( 


Quantity 


QTY: 


5 


Size 


SIZE: 


200X200 


Degree of Protection 


DOP: 


MDM 


Target Mobility 


TGT MOV: 


STAT 


Attitude 


ATT: 


10 


Method of Engagement 


METH ENG: 


AREA 


Weapon Category 


WCAT: 


AIR 


Weapon 


WPN: 


2000 


Ordnance 


ORD: 


BMBLT 


Fuze 


FZ: 


Q 


Distribution 


DISTRB: 


LAT 


Method of Fire 


MOF: 


CONT 


Method of Control 


MOC: 


FFE 


Observer Target 
Direction 


OT DIR: 


018 


Request Number 


RQNO : 


012116 


Date Time Group 


DTG: 


xxxxxxxx 
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MESSAGE TYPE 5 



Field Name Abbreviation 



Fire Order Universal 


FORDRM; 


From 


FR: 


To 


TO: 


Weapon Category 


WCAT: 


Target Number 


TGT NO: 


Request Number 


RQNO : 


UTM Coordinates 


GRID: 


Altitude 


ALT: 


Method of Control 


MOC: 


Method of Engagement 


METH ENG 


Distribution of Fire 


DISTRB: 


Time to Open Fire 


OPN FIR: 


Adjusting Unit 


UADJ: 


Number of Adjusting 
Rounds 


RDS: 


Adjusting Ordnance 


ORD: 


Adjusting Fuze 


FZ: 


Adjusting Lot 


LOT: 


Adjusting Charge 


CHG: 


Adjusting Angle 


ANGLE: 


Adjusting Time on 
Target 


TOT: 


Fire for Effect Unit 


UFFE: 


FFE No . of Rounds 


RDS: 


FFE Ordnance 


ORD: 



Requested Entry 

EClO 

AEOO 

ARTY 

BR2361 

390061 

DE65022500 

00020 

ADJ 

MRK/HA 

ZN 

0600 

G/3/10 

00003 

HE 

Q 

XY 

4 

26 

0640 

H/3/10 
00004 
I CM 
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Field Name 



Abbreviation 



Requested Entry 



FFE 


Fuze 


FZ; 


TI 


FFE 


Lot 


LOT: 


TW 


FFE 


Charge 


CHG: 


3 


FFE 


Angle 


ANGLE: 


16 


FFE 


Gun Target Line 


GTAZ: 


3200 


FFE 


Unit 


UFFE: 


1/3/10 


FFE 


No. of Rounds 


RDS: 


00006 


FFE 


Ordnance 


ORD: 


SMKRED 


FFE 


Fuze 


FZ: 


TI 


FFE 


Lot 


LOT: 


YX 


FFE 


Charge 


CHG: 


5 


FFE 


Gun Target Line 


GTAZ: 


1800 


FFE 


Unit 


UFFE : 


K/3/10 


FFE 


No. of Rounds 


RDS: 


00002 


FFE 


Ordnance 


ORD: 


WP 


FFE 


Fuze 


FZ: 


VT 


FFE 


Lot 


LOT: 


TE 


FFE 


Charge 


CHG: 


2 


FFE 


Angle 


ANGLE : 


24 


FFE 


Gun Target Line 


GTAZ: 


4000 


Date Time Group 


DTG: 


xxxxxxxx 
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APPENDIX C: DATA FOR PARAMETRIC ANALYSES 
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SPEED OF DATA ENTRY 
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222 


209 


253 
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I-H 

ro 


220 


266 


KD 

KO 

CN 


234 


O 

00 

CN 


209 


232 


307 


00 

CN 

CN 


238 


224 




















































i-H 


r- 


LD 


r- 


CN 


CN 


VD 






ro 


O 




<0^ 




o^ 


CN 


o 






< 




00 


r- 


in 


I-H 


\D 


a\ 


1 — 1 


CN 




C7^ 


r- 


ro 




VD 


ro 




UO 
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I-H 


CN 


CN 


CN 


I-H 


ro 


ro 




I-H 


I-H 


CN 


CN 


CN 


CN 


CN 


CN 






PQ 




i-H 


o\ 


r- 


<T\ 


LD 


I-H 


KO 


o 




r- 


I-H 


MO 


ro 


o\ 


I-H 


O 


UO 








r- 


00 


o 


CN 


ro 


C7^ 








CO 


C\ 




ro 




O 


CN 


ro 










I-H 




I-H 


I-H 


I-H 




I-H 


I-H 








I-H 


1 — 1 


I-H 


r-H 


I-H 


I-H 




vT 
















































00 


CN 


ro 






V£) 


LD 


LO 




CN 


00 


CN 


r- 


ro 


VD 


UO 


00 










00 


0\ 


I-H 


1 — 1 


LD 


I-H 


00 


LD 




00 


00 


VO 


ro 


CN 


I-H 


















I-H 


I-H 


1 — 1 


I-H 


I-H 


I-H 








I-H 


I-H 


I-H 


I-H 


I-H 


1 — 1 










r- 


00 


1 — 1 




ro 


CN 


ro 


CN 




I-H 


I-H 


CN 


CO 


1 — 1 


CN 


r- 


ro 






U4 




LD 


LD 


00 


r- 


CO 


in 


r- 


00 




LD 


VO 


CO 


r- 


I-H 


r- 


VO 


r- 




































I-H 








0) 










































a 

Sv. 




















































































H 










































0) 








r- 


CTi 


ro 


CN 


r- 




KD 


r- 




O 


ro 


UO 


r- 


O 


yo 




CN 






< 




LO 


00 


KD 


r- 


r- 


LO 


00 


r- 




LD 


LD 


00 


UO 


a\ 


UO 


r- 


r- 


CO 
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UO 


ro 


ro 


CN 


CN 


ro 








ro 










































ij 


















• t 






























































ci5 








































e 


















B 




















U1 






I-H 


CN 


ro 




in 


V£) 


r- 


00 


u 


I-H 


CN 


ro 




UO 


KO 


r-- 


CO 






O 


















0 








































tH 
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Eh 

2 ; 

w 

C 

Eh 

< 

Q 

Pm 

O 

u 

D 

U 

U 

< 






Message Type 


ID 


CQ 






1—1 




• — 1 












CN 


« — 1 




rH 






< 
















rH 


I — 1 


1 — 1 


00 




rH 




rH 


rH 




cri 




• — 1 






iH 


















rH 






vj' 


< 






« — 1 




1 — 1 


I — 1 


iH 






rH 














m 


m 




1 — 1 






1 — 1 












t — 1 


rH 








00 


<< 












i — 1 










rH 












<N 


m 


« — 1 


1 — 1 




«— 1 


rH 


« — 1 


iH 


1 — 1 


• — 1 
















< 




• — 1 


• — 1 




•H 


rH 


rH 


1 — 1 




rH 












rH 


1 


















m 


rH 


















c 


I — 1 
1 \ 


















CN 










rH 








1 W 

1 , 


n 

Format 


1 — 1 


i ^ 


m 




in 


KD 


r- 


CO 


Format 


rH 


CN 


00 




LO 


VO 


r^i 


00 
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- 



I 



# 

I 



0^ 

w 

< 

Q 

O 

u 

§ 

o 

u 

< 



CO 

p:; 

o 

p:: 

W 



0) 

Oi 

>1 

( 1 ) 

Cr» 

fd 

w 

w 

(D 

:s: 



LO 



ro 



CN 



m 



m 



pp 



PQ 



CQ 



CO 



ro 

4J 

fd 

e 

o 

pLl 



CN 



ro 



LO 



VO 



CO 



•4-> 

cd 

e 

u 

o 

U-i 



CN 



00 



in 



CN 



CN 



VO 



00 
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APPENDIX D: NORMALIZED TEST DATA 



X 

:z: 

w 

c 

< 

Q 

Cm 

O 

Q 

pa 

pq 

04 

w 











O 


V£) 


iH 


cH 






o 


m 




r- 


in 


00 


tH 


r- 


CP 


00 


CP 












O 


00 


CN 


in 




I — 1 


m 




VO 


ro 


CP 


00 


tH 


o 


in 


o 






CQ 




00 


00 


1 — 1 


CN 


ro 


o 


ro 


rH 




00 


o 


00 


tH 




o 


m 


m 














rH 


rH 


rH 


rH 


rH 


rH 






tH 




tH 


tH 


tH 


tH 


rH 




u J 
















































ro 


00 


O 


<y\ 




r- 


rH 


CP 








VO 


CN 


r' 


r- 


O 


tH 










00 


in 


in 


00 


00 


CN 


VO 


tH 




m 


CN 


00 


CP 


VO 


rH 


in 


ro 






< 




00 


CP 


o 


ro 


in 


CN 


o 


O 




rH 


CN 


o 


tH 


VO 


CN 


CP 


VO 














cH 


iH 


rH 


rH 


tH 


rH 




rH 


tH 


rH 


tH 


tH 


rH 


rH 


tH 










V£> 




VD 


rH 


in 


00 


CJ\ 


VO 




r- 


ro 


in 


VO 


rH 




O 


00 










ro 


KO 




ro 


VO 


CN 




VO 




ro 


CP 


VO 


O 


CN 


CP 


ro 


CP 






CQ 




CP 


o 




CN 


m 


O 


ro 


CN 




O 


o 


o 


CN 




r- 




m 












»H 


iH 


cH 


1 — 1 


tH 


rH 


rH 




rH 


rH 


rH 


rH 


rH 




tH 


tH 




vl' 








m 


tH 


CP 




<y\ 


in 


in 




CN 


in 


ro 


CN 




VO 


00 


tH 










ro 




m 






CJ\ 


CN 


rH 




tH 


VO 


ro 


O 


CN 


CP 


o 








< 






CT> 


C7\ 


ro 




VO 


o 


tH 






r- 


ro 




00 


tH 


VO 


CP 
















1 — 1 


rH 


rH 


1 — 1 


tH 




rH 


tH 


rH 


rH 


tH 


tH 


rH 


rH 












ro 


o 


ro 


O) 


in 


rH 


VO 




VO 


O 


VO 


m 


ro 


in 


00 


ro 






CO 




00 


o 


ro 




G\ 




r- 


ro 




CT\ 




VO 




ro 


m 


rH 


tH 








00 


CP 


o 


ro 


CN 


o 


ro 


o 




CO 


CP 


00 


o 




CP 




ro 


Q) 

CU 

*s. 










nH 


cH 


rH 


rH 


rH 


rH 


rH 










rH 


rH 




rH 


rH 




1 ’ 1 


















































































(D 








I— 1 


KD 


r- 


O 


O 


VO 


CN 








VO 


VO 


00 


ro 




o 


ro 








r- 


iH 




O 


O 


in 


ro 


<Tv 




00 


tH 


VO 


CN 


o 


CP 


in 


in 


CP 

fd 








00 


O 


o 


O 


in 


rH 


o 


O 




CP 


O 


CN 


ro 


CN 




CN 




w 










iH 


CN 


rH 


rH 


rH 


rH 


rH 






tH 


rH 


rH 


rH 




tH 




w 










































UJ 










































s 








LO 


m 


O 


O 


in 


O 


O 


O 






CN 


00 


O 




00 




in 






CO 






CN 


O 


O 




O 


o 


o 




in 




tH 


O 


CP 


tH 




o 










o 


00 


in 


CN 


O 


tH 






rH 


00 




o 






o 


CN 




























• 


• 


• 


• 


CP 


• 


• 


• 




CN 








iH 




rH 


rH 


rH 


rH 


rH 




rH 






rH 


• 




i— 1 


tH 








in 


in 


o 


in 


rH 


rH 


O 


O 




in 


o 


m 


O 


in 


m 


m 


in 








< 




CN 




o 


rH 


VD 




in 


in 




CN 


in 


CN 


O 




CN 


CN 


CN 








Os 


CP 


o 




in 


rH 








CP 


o 


o 


ro 


tH 


CP 




iH 














rH 


rH 


CN 


iH 


1 — 1 


tH 






rH 


rH 


rH 


rH 




tH 


ro 














O 


CO 


00 


C7\ 


in 


O 




CN 


CN 




rH 


in 




(P 


00 












in 


O 


ro 


o 


VO 


rH 


O 




m 


in 


VO 


rH 


00 


o 


in 












m 


\D 


O 


o 


ro 


r- 


rH 


O 




00 


CO 


VO 


tH 


rH 




CN 


tH 




, 1 










rH 


rH 


rH 




rH 


rH 










rH 


rH 


tH 


tH 


tH 










i-H 


CP 


00 


in 


00 


\o 


CN 


in 




r- 


tH 


CN 


00 


VO 


tH 


ro 












CO 


VD 


O 


rH 


o 




VO 


rH 




o 


rH 


in 




CN 


00 


<p 


o 






< 




r-* 


r- 


00 


1 — 1 


ro 


00 


CP 


rH 




^ ! 


I — 1 


00 


rH 


CP 




in 










rH 








1 — 1 


rH 






rH 


CN 


tH 


ro 




rH 


CN 


tH 


rH 


tH 








1 \ 


















1 1 






































































































a 


















a 
















00 




cn 




u 


rH 


CN 


ro 




in 


VO 


r- 


00 


u 


tH 


CN 


ro 




in 


VO 








0 


















0 






















Cm 


















Cm 
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V 






SPEED OF DATA ENTRY 











CN 


1— 1 


00 


o 


o 


ro 




G^ 






f — 1 


in 


in 


CN 


CD 


CN 


cn 












00 


00 




CD 


ro 








in 


CD 


<y\ 


o 


CD 


CD 




o 






PQ 




O 




«H 


o 




o 


CN 


CN 




o 


CN 


cn 


fH 




o 


o 


o 










iH 




iH 


1 — 1 


1 — 1 


fH 


f — 1 


fH 




fH 


fH 




fH 


fH 


.H 


fH 


fH 




















































CD 


r- 


CN 














in 


CD 




CD 


fH 


CO 




cn 












CN 


CT» 


fH 


CN 


a^ 




1 — 1 




G> 


n- 


fH 


00 


r- 


ro 




00 






< 




00 


00 


nH 


O 


CN 


00 




in 




cn 


00 


fH 


fH 


CN 


fH 


CN 


CN 














rH 


iH 


fH 




fH 


f — 1 








•H 


iH 


fH 


fH 


fH 


fH 










cn 








O 


ro 


CN 


CD 




ro 


o 


CN 


fH 


O 


in 


fH 


CN 










00 


CN 


o^ 


CT^ 


in 




in 


G^ 




1 — 1 


in 


in 


ro 


00 


ro 


CN 


CD 






CQ 




in 


00 




iH 


CN 


00 


ro 


CN 




00 


00 


ro 


CN 


ro 


G\ 


f — 1 


CN 










•H 






fH 


fH 




fH 


f — 1 








fH 


1 — 1 


f — 1 




f — 1 


• 

1 1 






















































o 




CN 


ro 


r- 


G\ 


f — 1 






ro 


G^ 


CN 




cn 


CN 


fH 










CN 


CD 


ro 






in 


CN 


00 




o 


CD 


ro 


ro 


fH 


in 


CN 


in 






< 




n- 


n- 


a^ 


a\ 


CN 


cn 


in 


CN 




00 


00 


ro 


fH 


O 


cn 






















fH 




f — 1 


fH 








f — 1 


fH 


fH 




fH 


fH 










cr» 


in 


CD 


CD 


a^ 




r-- 


ro 




fH 


o 


ro 


00 


O 


f — 1 


O 


o 






PQ 






ro 


o 


CN 


ro 


ro 




CN 




CD 


iH 


CN 


in 


cn 


CD 


O 


cn 








<T\ 


cr\ 


ro 


CN 


ro 


00 


fH 


ro 




r- 


cn 


ro 


CN 




fH 


O 


o 


0) 

Cu 


ro 










iH 


fH 


fH 




fH 


fH 








fH 


f — 1 


f — 1 


fH 


fH 


fH 










































Eh 










































Q) 








CJ^ 


in 


CD 


• fH 


CN 


fH 


n' 


•CN 




fH 


00 


fH 


cn 


CN 


ro 


CD 


in 








• — 1 


ro 


iH 


CD 






00 






00 


CN 




fH 


in 


o 


in 


CN 


0^ 




< 




a^ 




O 


fH 


CN 


00 


ro 


CN 






00 


ro 


G^ 




G^ 


fH 


fH 


01 












»— 1 


fH 


fH 




fH 


fH 








fH 




fH 




fH 


fH 


0) 
<1 V 










































U/ 


















































00 


00 


CN 


in 


CD 


in 


G^ 


fH 




00 




CN 


CD 


CD 


CD 


CD 


cn 






PQ 




1— 1 


iH 


o^ 




CN 


f — 1 




ro 




fH 


CD 


r- 


CN 


CN 




in 


r- 








n- 


n- 


CD 




O 


CD 


fH 


CN 




r- 


CD 


00 


O 


O 


00 


CN 


f — 1 




CN 














fH 




fH 


fH 










f — 1 


fH 




fH 


fH 








00 


in 






in 


fH 


fH 


00 




o 


in 






O 


CD 


in 


O 












iH 


G\ 






00 


CN 


ro 


o 




in 


r- 


r' 


r-' 


O 




CN 


in 






< 










G\ 


ro 


00 


CN 


ro 




CD 


in 


cn 


o 


o 


00 


o 


o 


















iH 




fH 


fH 










fH 


fH 




fH 


fH 










00 


•H 


in 


O 


CN 




CXD 


CN 




CN 


CN 


CN 


CN 




in 


r- 


ro 










o 


ro 


00 


O 


CD 


CD 


ro 


(GT 




cn 


G^ 


cn 


cn 


in 


00 


n- 


CN 






PQ 




00 




00 


O 


<J\ 


r- 


o 


fH 




CD 


CD 


fH 


fH 


fH 


00 


o 






1 












iH 






f — 1 


f — 1 








fH 


fH 


fH 




fH 


fH 










CN 




»H 


ro 






in 


00 




cn 






f — 1 


G^ 


ro 


in 


o 










CD 


CD 


ro 


CN 


in 


in 


fH 


ro 




CD 


in 


r- 


ro 


CD 


CN 


f — 1 


o 






< 




cr> 


r- 


r- 




fH 


CD 


fH 


in 




r- 


CD 


o 


CN 


CN 


cn 


fH 


in 








ro 






fH 




fH 




fH 


H 








CN 


f — 1 


fH 




fH 


fH 








11 


















1 t 
















































































C3 






















e 


















a 
















00 




CO 






1 — 1 


CN 


ro 




in 


CD 


r- 


CO 


u 


f — 1 


CN 


ro 




in 


CD 


r-- 






0 


















0 






















Cl, 


















Cn 
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m . 4 . - 4 * 




0 



\ 



1 





:s; 

w 

c 

< 

Q 

O 

C 

D 

U 

U 

c 



CO 

§ 

w 













00 




00 












V£> 


00 




00 










CO 


o 


O 


(N 


o 


CM 


o 


o 


o 


o 


o 




CN 


o 


CN 


o 


o 












o 




o 












o 


o 




o 


- 














• 
















• 






• 
































































00 


00 


00 


a\ 




00 




00 


00 






< 


O 


o 


o 


o 


o 


o 


o 


osl 


CN 


CN 


VD 


o 


CN 


o 


CN 


ON 






















o 


o 


o 


o 




o 




o 


o 






















• 


• 


• 


• 




• 




• 












00 






00 


















00 










CO 


o 




o 


o 




o 


o 


o 


o 


o 


o 


o 


o 




o 


o 










o 






o 


















o 














• 






• 


















• 






















































00 




00 


00 


00 






00 


















<r! 


o 


o 




o 








o 


o 




o 


o 


o 


o 


o 


o 












o 




o 


o 


o 






o 
























• 




• 


• 


• 






• 






















ro 






00 












00 


00 








a\ 










00 






00 












00 


00 














P-] 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


CN 










• 


















• 


• 








• 


0) 






































a 

»v^ 












































































Eh 






































0) 
















00 










00 












tn 




< 


o 


o 


o 


o 


o 


00 


o 


o 


o 


o 


00 


o 


o 


o 


o 


o 


»u 
















o 










o 












w 


• 




































w 

(1^ 












































































s 












































ro 


00 




00 


00 


00 


00 


00 


00 


























o 
































CO 


r— 1 


1— 1 




•H 


I — 1 


1 — 1 


t — 1 


1 — 1 


i-H 


o 


o 


o 


o 


o 


o 


o 




CN 




• 


• 




• 




• 


• 


• 






























































00 


00 




00 


00 


00 


00 




00 












00 














o 










o 




o 


o 


o 


o 


o 








< 


o 


1 — 1 


1 — 1 




1 — 1 


•H 


I — 1 


fH 




1 — 1 












r-H 










• 


• 




• 


• 


• 


• 




• 












• 
























uo 


















1 


CO 


o 


o 


o 


o 


o 


o 


o 


o 


CN 

• 


o 


o 


o 


o 


o 


o 


o 
































uo 












< 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


CN 


o 


o 


o 








t — 1 


















CN 










• 














1 1 


















1 1 






























































fO 


















fd 






















E 


















a 




















CO 




U 


1 — I 


<N 


00 




uo 


VD 




00 


u 


1 — ! 


CN 


00 




uo 


yo 




00 






0 


















0 






















Cn 


















tl 
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« 

w 

< 

< 

Q 

O 

X 

u 

D 

O 

U 

< 



cn 

o 

W 



Message Type 


in 


PQ 


0 


0 


0 


0 


0 


0 


0 


0 


.023 


.023 


0 


0 


0 


0 


. 023 


0 


c 


0 


0 


.023 


0 


0 


0 


.023 


0 


0 


.023 


0 


0 


0 


0 


0 


0 




pp 


CO 

0 

• 


0 


0 


0 


0 


CO 

0 

• 


0 


0 


0 


0 


0 


0 


0 


.096 


0 


0 




< 


0 


0 


.048 


0 


0 


0 


CO 

0 

• 


CO 

0 

• 


0 


0 


0 


0 


0 


oc 

0 

• 


0 


0 


m 


CQ 


0 


0 


0 


0 


0 


m 

CO 

0 

• 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


C 


0 


.083 


.083 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


99T* 


0 


o’ 


CNJ 


PP 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


< 


0 


0 


0 


0 


0 


0 


0 


0 ! 


. 143 


0 


0 


0 


0 


0 


0 


0 


1 


PP 


.25 


0 


0 


0 


0 


0 


0 


• 

0 


0 


0 


0 


0 


0 


0 


0 


0 




< 


rn 

1 1 


0 


0 


.25 


0 


0 


0 


0 


0 


1 \ 


0 


0 


0 


0 


0 


0 


0 


0 




1 

Formal 


iH 


(N 


ro 




m 


VO 


r- 


00 


1 

Format 


iH 


(N 


cn 




m 


VO 


r-* 


00 
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I 




I 




APPENDIX E: INTERACTION TABLES 



SPEED OF DATA ENTRY 



AXBXC 



C/B 


1 


2 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


1.308 


2.400 


1.758 


1.870 


1.723 


2.259 


2.079 


1.880 


2.449 


2.024 


2 


1.423 


2.000 


2.919 


2.039 


1.764 


3.963 


1.922 


1.956 


2.858 


2.262 


3 


1.808 


1. 800 


3.077 


2.747 


2.231 


1.519 


1.743 


2.132 


2.398 


1.984 


4 


2.153 


2.915 


2.343 


2.600 


2.610 


2.259 


2.300 


2.403 


2.608 


2.373 


5 


2.616 


3.836 


2.799 


3.342 


2.940 


4.111 


2.124 


2.636 


3.245 


3.384 


6 


1.615 


2.171 


2.231 


2.727 


2.271 


2.888 


1.643 


1.752 


1.990 


2.226 


7 


2.077 


2.550 


2.403 


2.374 
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APPENDIX F: NONPARAMETRIC ANALYSIS 

EASE OF USE 
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SUITABILITY 



Sub j . 


1 


2 


3 


4 


S 


R 


S 


R 


S 


R 


S 


R 


1 


4 


1 


6 


4 


5 


2.5 


5 


2.5 


2 


8 


1 


14 


4 


9 


2 


11 


3 


3 


15 


3.5 


15 


3.5 


0 


1 


5 


2 


4 


13 


3 


14 


4 


9 


1 


11 


2 


5 


9 


3 


14 


4 


3 


1 


4 


2 


6 


10 


3.5 


10 


3.5 


1 


1 


5 


2 


7 


5 


3 


5 


3 


5 


3 


0 


1 


8 


5 


1.5 


15 


4 


5 


1.5 


10 


3 



8 19.5 30 13 17.5 



X 2 = 3(41^5) [380.25 + 900 + 169 + 306.25] - 3(8) (5) 

= 11.64* 
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